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EXECUTIVE  SUMMARY 

Providing  reduced  life-cycle  cost  and,  at  the  same  time,  improving  operational 
availability  are  fundamental  goals  of  the  Performance  Based  Logistics  (PBL)  and  other 
logistics  initiatives  of  the  U.S.  Department  of  Defense.  In  many  PBL  contracts,  the 
contractual  arrangements  are  typically  stipulated  at  the  level  of  individual  components 
(such  as  a  fuel  cell)  or  a  logistic  element  (such  as  inventory  of  certain  spare  parts). 
While  achieving  component  level  performance  goal  is  certainly  important,  what  really 
matters  to  a  war  fighter  is  the  operational  availability  of  the  weapon  system.  Hence, 
there  is  a  need  to  develop  a  methodology  and  an  apparatus  for  estimating  the 
operational  availability  (Ao)  of  a  weapon  system  based  on  the  component-level 
reliability  and  maintainability  data.  This  current  research  is  aimed  at  this  need. 

Specifically,  we  present  two  spreadsheet  models  and  one  discrete-event 
simulation  model  using  Arena  simulation  language.  The  first  model  primarily  supports 
life  cycle  cost  calculations,  but  ignores  the  interactions  among  reliability,  time  to  repair, 
and  operational  availability.  The  second  model,  while  it  does  address  these  basic 
interactions,  does  not  consider  the  full  range  of  life  cycle  costs.  However,  both  the  first 
and  the  second  model  are  static  -  they  can  only  support  average  case  analyses  and 
sensitivity  analyses.  The  third  model  incorporates  the  interactions  among  reliability, 
time  to  repair  and  operational  availability  into  a  simulation  model  that  can  support  a 
weapon  system  level  risk  analysis.  In  their  current  form,  these  models  are  developed 
as  a  proof-of-concept.  That  is,  we  are  not  presenting  a  research  case  involving  field 
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data,  but  rather,  demonstrating  the  potential  methodology  and  a  tool  using  hypothetical 
yet  realistic  data. 


I.  INTRODUCTION 


The  US  Department  of  Defense  is  engaged  in  a  number  of  management 
initiatives  (related  to  weapon  system  logistics  and  support)  intended  to  provide  reduced 
lifecycle  cost  while  simultaneously  improving  operational  availability,  Ao.  Performance- 
based  Logistics  (PBL)  is  one  such  program  that  entails  the  establishment  of  a  particular 
kind  of  contractual  vendor-client  relationship  between  a  logistic-service  provider  and  a 
weapon-system  manager.  The  Quadrennial  Defense  Review  mandated  the  DoD 
implement  PBL  in  order  to,  “compress  the  supply  chain  and  improve  readiness  for  major 
weapons  systems  and  commodities”  (OSD,  2001 , 56).  A  key  aspect  of  PBL  contracts  is 
their  outcome  focus;  the  client  organization  is  supposed  to  specify  key  performance 
goals  and  allow  the  vendor  to  determine  the  best  way  of  obtaining  those  goals  (ASN- 
RDA,  2003). 

This  paper  will  not  re-examine  the  core  questions  of  whether  PBL  works,  or  why 
it  works,  as  those  questions  have  been  examined  extensively  elsewhere  (e.g., 

Berkowitz  et  al. ,  2003).  Rather,  we  take  as  our  starting  point  the  question  of  how  best 
to  value  the  desired  outcomes  of  a  PBL  contract.  After  all,  as  contractual  vehicles,  the 
price  of  the  services  to  be  provided  must  be  negotiated.  Also,  given  a  limited  budget 
but  a  proactive  program  manager,  there  will  always  be  more  opportunities  to  improve 
logistical  support  for  a  weapon  system  than  dollars  available  to  fund  those  opportunities. 

We  assume  that  opportunities  to  improve  logistics  outcomes  should  be  valued  on 
the  basis  of  the  cost-effectiveness  of  those  opportunities.1  As  in  the  private  sector,  the 
cost  effectiveness  of  an  opportunity  (investment)  is  its  mission-value-over-time  (profit,  in 
the  case  of  the  private  sector)  divided  by  its  cost-over-time.  It  would  thus  be  a  mistake 
to  take  the  cost  differentials  of  various  logistic  service  alternatives  as  a  statement  of 
value  because  cost  in  no  way  informs  the  value  of  that  service  to  the  weapon-system 


1  Caplice  and  Sheffi  (1994),  in  reviewing  a  panoply  of  logistics  metrics,  categorized  metrics  based  solely  on 
comparisons  of  inputs  (such  as  cost  comparisons)  as  utilization  metrics,  while  they  categorized  comparisons  of 
outputs  per  input  (such  as  what  we  are  calling  cost-effectiveness)  as  productivity  metrics.  They  made  the  point  that 
utilization  measures  are  usually  related  to  process  (as  opposed  to  performance)  management. 
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operator.  Even  if  one  is  willing  to  assume  that  current  expenditures  are  cost  effective 
(and  hence,  any  cost  reduction  would  be  even  more  cost  effective),  there  is  no  way  to 
assess  one  alternative  against  another  without  a  direct  measure  of  value;  mere  cost 
differentials  ignore  the  fact  that  the  alternatives  may  have  different  impacts  on  mission 
value. 


We  will  further  assume  that  the  mission  value  of  a  logistical  service  is  a  function 
of  weapon-system  performance,  as  neither  a  weapon-system  component  (such  as  a 
fuel  cell)  nor  a  logistic  element  (such  as  spares  inventory)  can  contribute  to  mission 
objectives  except  through  the  weapon  system.  From  a  warfighter’s  viewpoint,  a  weapon 
system  is  either  capable  of  supporting  a  mission,  or  it  is  not.  While  a  fuel  cell  may  be  a 
necessary  condition  for  the  system  to  be  mission  capable,  it  is  not  a  sufficient  condition. 

Operational  availability  (Ao)  is  a  primary  metric  used  to  determine  the  probability 
that  a  weapon  system  will  be  capable  of  supporting  a  mission.  For  example,  in  an 
aircraft  squadron,  Ao  of  85%  implies  that  an  average  of  85%  of  the  aircraft  will  be 
available  to  fly  in  support  of  some  mission  objective.  Goals  are  often  stated  for  Ao 
levels,  and  mission  planning  must  take  Ao  into  account.  Moreover,  neither  a  warfighter 
nor  a  resource  manager  wanting  to  make  contingency  plans  should  be  content  with 
knowing  the  nominal  (target)  or  the  average  Ao  level.  He  or  she  should  have  a  sense  of 
the  distribution  of  Ao  around  the  target  levels:  the  probability  that  Ao  will  fall  below  some 
critical  level. 

It  is  also  possible  to  measure  Ao  for  fuel  cells,  as  well  as  aircraft;  an 
improvement  in  Ao  for  the  fuel  cell  will  provide  at  least  some  marginal  improvement  in 
Ao  for  the  aircraft.  But,  this  improvement  will  not  be  one-to-one;  large  improvements  in 
fuel-cell  availability  may  yield  only  trivial  improvements  in  aircraft  availability,  depending 
not  only  on  the  failure  rate  of  the  fuel  cells,  but  on  the  performance  and  availability  of  all 
the  other  critical  components  of  the  aircraft.  Likewise,  better  fuel-cell  availability  will 
reduce  the  risk  that  a  particular  weapon  system  will  not  be  operational  for  a  particular 
mission,  but  the  magnitude  of  that  risk  reduction  depends  on  the  probability  that  all  the 
other  critical  components  of  the  aircraft  are  available. 
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Hence,  the  value  of  an  improvement  of  component  logistics  can  only  be 
understood  in  terms  of  the  performance  of  all  the  other  critical  components  of  a  weapon 
system.  Similarly,  the  value  of  an  improvement  in  a  single  logistics  element  (such  as 
spares  inventory)  can  only  be  determined  in  conjunction  with  other  key  logistics 
elements. 

The  modeling  approach  we  will  outline  in  this  paper  has  applicability  beyond 
PBL.  It  is  useful  in  understanding  the  value  of  component-level  logistic  services,  or 
services  directed  at  only  a  subset  of  logistic  elements  (inventory  only,  or  depot-level 
repair  only).  However,  we  contend  that  an  implementation  of  PBL  that  is  fully  consistent 
with  the  original  intent  of  performance-based  service  acquisition  must  use  an  approach 
similar  to  the  one  we  outline,  because  it  is  impossible  to  put  a  value  (and,  hence,  a 
contract  price)  on  those  services  without  such  an  approach. 
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II.  LITERATURE  REVIEW 


While  we  are  arguing  for  an  assessment  of  value  that  will  provide  a  more 
complete  picture  of  the  cost  effectiveness  of  a  PBL  proposal  (by  providing  a  numerator 
to  a  productivity  ratio),  we  recognize  that  an  estimation  of  the  lifecycle  costs  of  such 
proposals  is  far  from  trivial.  Outsourced  logistic  services  for  weapon  systems  are 
particularly  difficult  to  cost;  for  example,  the  ongoing  contract  management  (transaction) 
costs  can  be  substantial,  but  are  rarely  measured  (Domberger,  Jensen  &  Stonecash, 
2002). 

We  think  such  transaction  costs  are  particularly  important  in  light  of  a  recent 
Government  Accountability  Office  report  (GAO,  2004)  that  was  critical  of  systems-level 
PBL  contracts;  this  document  recommended  greater  emphasis  on  PBL  contracts  at  the 
component  level,  especially  for  commodity-type  components  (which,  according  to  the 
GAO,  reflected  “commercial  best  practices”).  PBL  contracts  on  commodities  would  be 
especially  appealing  because  vendors  providing  commodities  can  expect  to  enjoy 
economies  of  scale  that  the  DoD  could  not  experience  (as  vendors  would  be  able  to 
offer  those  commodities  across  a  broad  population  of  users).  These  increased 
economies  of  scale  would  reduce  the  price  of  such  services.  Unfortunately,  of  course, 
aside  from  domestic  transportation  and  depot-level  spares  for  a  relatively  small  set  of 
components  used  commonly  between  defense  and  industry,  the  number  of  critical 
components  (or  logistics  elements)  of  weapon  systems  that  can  be  considered 
commodities  is  relatively  small.  For  non-commodity  items,  a  key  economic 
consideration  in  out-sourcing  is  the  increase  in  transaction  costs  entailed  by  dealing 
with  an  outside  vendor  (Gufstafson  et  al.,  1996).  Such  costs  increase  substantially 
when  one  is  offering  a  PBL  contract  at  the  component  level.  As  we  will  show,  aside  from 
the  additional  burden  of  contract  maintenance  for  many  small  contracts,  the  proper 
valuation  and  management  of  such  component-level  contracts  entails  the  development 
of  a  comprehensive  model  which  incorporates  key  performance  dimensions  of  all  critical 
components. 
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Perhaps  in  an  effort  to  reduce  such  transaction  costs,  or  perhaps  in  response  to 
a  complaint  that  PBL  involved  too  many  metrics,  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology  and  Logistics  (USD-ATL,  2004)  recently  issued  guidance  for 
PBL  metrics.  While  clearly  indicating  that  PBL  could  be  applied  at  the  subsystem  or 
major  assembly  level,  the  memo  listed  five  key  performance  criteria:  1 )  weapon  system 
operational  availability,  2)  weapon  system  operational  reliability,  3)  weapon  system  cost 
per  usage,  4)  logistics  footprint  for  a  weapon  system,  and  5)  response  time  required  for 
weapon  system  logistics  support. 

Of  course,  these  measures  are  interrelated.  We  think  the  central  non-cost 
measure  is  operational  availability.  The  other  three  non-cost  measures  can  all  be  seen 
in  some  ways  as  subsidiary  to  availability.  Reliability  (e.g.,  time  to  failure),  footprint 
(e.g.,  number  of  spares  and  size  of  fielded  or  intermediate  maintenance  and  repair 
facility)  and  response  time  (e.g.,  time  to  repair)  are  all  critical  determinants  of 
availability.  Yet,  there  may  be  good  reasons  to  measure  reliability,  footprint  and 
response  time  separately.  For  example,  reliability  affects  not  only  availability,  but  also 
the  probability  of  system  failure  in  the  field;  likewise,  footprint  affects  not  only 
availability,  but  operational  agility  as  well.  However,  operational  availability  in  many 
ways  summarizes  reliability,  response  time  and  footprint.  We  will  develop  a  model  in 
the  next  section  that  demonstrates  the  precise  interaction  between  time  to  failure,  time 
to  repair,  and  spare  inventory  levels.  It  also  demonstrates  how  these  variables 
determine  availability.  Thus,  as  they  affect  Ao,  footprint,  response  time,  and  even 
reliability  are  all  process  and  not  performance  measures.  We  will  focus  on  availability 
(with  the  caveat  that  it  may  not  be  the  sole  determinant  of  value)  because  it  is 
necessary  to  an  understanding  of  value. 

In  specifying  performance  outcomes  (but  not  processes)  to  a  vendor,  PBL 
contracts  are  deliberately  designed  to  transfer  some  degree  of  operational  and  financial 
risk  to  a  vendor  (Doerr,  Lewis  &  Eaton,  forthcoming).  As  risk  transfer  is  an  intended 
outcome  of  the  initiative,  and  as  the  risk  of  falling  below  a  certain  level  of  operational 
availability  is  an  important  performance  dimension,  it  is  clearly  important  to  incorporate 
the  risk  associated  with  operational  availability  at  the  system  level  into  a  measure  of 
value.  From  the  warfighter’s  point  of  view,  this  risk  may  be  the  key  performance 
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dimension  (Eaton,  Doerr  &  Lewis,  forthcoming).  The  warfighter,  after  all,  is  less 
concerned  with  the  average  number  of  mission-capable  aircraft  than  he  is  concerned 
with  the  probability  that  he  will  have  enough  aircraft  to  fly  a  particular  mission.  The 
procedure  we  will  outline  allows  the  assessment  of  a  proposed  logistics  improvement 
not  only  on  the  average  impact  that  improvement  would  have  on  the  operational 
availability  of  the  aircraft,  but  on  the  risk  associated  with  the  operational  availability  of 
the  aircraft  as  well. 

Weapon  systems  are,  of  course,  the  military’s  key  capital  assets  related  to 
operational  capacity,  and  the  logistics  services  in  question  can  be  seen  as  primarily 
affecting  the  level  of  operational  capacity  available  to  the  warfighter.  The  sort  of  risk 
measurement  we  are  proposing  is  increasingly  recognized  as  central  to  the  valuation  of 
operational  capacity  of  corporate  assets  in  the  private  sector  as  well. 

Assessments  of  risk/return  profiles  for  capital  assets  are,  of  course,  behind  the 
recent  work  on  Real  Options  (Mun,  2002).  And  in  capacity  planning  in  particular,  the 
incorporation  of  risk  into  capacity  models  was  listed  in  a  recent  literature  review  as  a 
key  area  in  which  research  was  expected  to  develop  (Van  Miegham,  2003).  Risk-based 
models  have  recently  been  applied  to  the  acquisition  of  production  capacity  for  airfoils 
used  in  military  aircraft  (Prueitt  &  Park,  2003).  Mostly,  risk-based  capacity  models  deal 
with  technological,  demand,  or  price  uncertainty,  and  are  not  directly  applicable  to  the 
valuation  of  logistic  services  and  the  uncertain  impact  those  services  will  have  on 
system  availability  (capacity).  The  point  we  are  making  is  that  there  is  growing 
consensus  that  a  proper  valuation  of  capacity-related  planning  (such  as  the  planning 
associated  with  offering  a  PBL  contract)  must  include  an  assessment  of  risk. 

In  this  paper  we  develop  three  models  as  decision-support  systems  (Keen  & 
Morton,  1978;  Power,  2002;  Turban  &  Aronson,  1998).  The  term  “decision-support 
system”  implies  use  of  computer-based  systems  to: 

1 .  assist  the  warfighters  in  their  decision  process  in  semi-structured  tasks, 

2.  support,  rather  than  replace,  the  warfighter’s  judgment,  and 

3.  improve  the  effectiveness  of  the  practical  decision-making  process. 
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The  dramatic  improvements  in  computer  power  and  software  capability  (such  as 
spreadsheet  and  simulation  models)  allow  convenient  access  to  powerful  decision- 
support  systems  for  improved  decision  making.  Making  such  models  available  as 
decision-support  systems  is  the  primary  goal  of  this  research. 
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III.  MODELS 


In  this  section,  we  present  two  spreadsheet  models  and  one  discrete-event 
simulation  model  using  Arena  simulation  language  (Kelton,  2004).  The  first  model 
primarily  supports  lifecycle  cost  calculations  but  ignores  the  interactions  among 
reliability,  time  to  repair,  and  operational  availability.  The  second  model,  while  it  does 
address  these  basic  interactions,  does  not  consider  the  full  range  of  lifecycle  costs. 
However,  both  the  first  and  the  second  model  are  static — they  can  only  support  average 
case  analyses  and  sensitivity  analyses.  The  third  model  incorporates  the  interactions 
among  reliability,  time  to  repair  and  operational  availability  into  a  simulation  model  that 
can  support  a  risk  analysis,  but  which  does  not  directly  address  lifecycle  cost  issues. 

In  their  current  form,  these  models  are  intended  as  a  proof-of-concept  only.  That 
is,  we  are  not  presenting  a  research  case  involving  field  data;  rather,  we  are 
demonstrating  the  potential  of  an  approach  using  hypothetical  data. 

3.1.  Spreadsheet  Lifecycle  Cost  Model  (Model  1) 

Model  1  is  a  compressive  lifecycle  cost  analysis  model  for  a  hypothetical  UAV 
(unmanned  aerial  vehicle)  case  study  intended  as  a  proof-of-concept  for  our  modeling 
approaches.  This  case  study  was  adapted  from  Logistics  Engineering  class  lecture 
notes  at  the  Naval  Postgraduate  School  (Kang,  2004).  The  complete  case  study  is 
described  in  Appendix  A,  and  the  spreadsheet  model  is  available  from 
http://web.nps.navy.mil/  ~mn4310/UAV_Model_1  .xls. 

This  model  computes  the  total  system  lifecycle  cost  for  major  weapon  systems 
from  R&D  to  deployment  to  phase-out.  The  lifecycle  cost  includes  research, 
development,  test  and  evaluation,  acquisition,  production,  operations  and  maintenance, 
and  phase-out  costs.  This  model  is  a  comprehensive  decision-support  tool  for  program 
managers.  The  model  can  be  used  to  establish  the  baseline  total  ownership  cost  of 
major  weapon  systems  during  the  planning,  as  well  as  operations,  stages.  The  user 
can  conduct  sensitivity  analyses  on  various  input  parameters  such  as  reliability, 
manning,  training,  and  R&D  cost.  As  the  user  changes  any  of  the  parameters,  the 
model  immediately  updates  the  total  lifecycle  cost,  so  the  user  can  see  the  financial 
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impact  of  input  parameter  changes  in  the  long-run.  We  suggest  the  reader  download  the 
spreadsheet  model  and  change  some  of  the  parameters  in  the  “INPUT”  worksheet. 

3.2.  Revised  Spreadsheet  Model  (Model  2)  and  Simulation  Model  (Model  3) 

A  shortcoming  of  the  spreadsheet  model  (Model  1 )  is  that  it  cannot  analyze  the 
dynamic  relationship  between  reliability  and  operational  availability.  For  example, 
deterioration  in  reliability  of  a  certain  component  will  decrease  the  system’s  operational 
availability.  At  the  same  time,  the  workload  at  a  repair  shop  will  increase,  forcing  the 
repair  turnaround  time  to  become  longer,  which  in  turn  will  decrease  the  operational 
availability  of  the  system.  In  Model  1 ,  the  average  repair  turnaround  time  remains  the 
same  regardless  of  the  changes  in  component  reliability. 

To  overcome  this  limitation,  we  have  developed  a  discrete-event  simulation 
model  (Model  3)  that  can  be  used  along  with  a  revised  spreadsheet  model  (Model  2). 
Model  2  is  essentially  derived  from  Model  1 .  It  is  a  small-scale  spreadsheet  model 
designed  to  focus  on  reliability  and  maintainability.  Given  logistics  input  parameters  (see 
Figure  1 ),  Model  2  computes  spare-parts  requirements,  inventory,  transportation  and 
repair  costs  followed  by  the  total  maintenance  costs  over  the  lifecycle  of  the  system. 
Model  2  does  not  consider  R&D  cost  or  infrastructure  costs.  It  only  considers  variable 
costs  while  operating  the  weapon  system.  Figure  2  shows  the  total  lifecycle 
maintenance  cost  of  $442,656,976  based  on  the  input  parameters  in  Figure  1 .  To 
demonstrate  how  Model  2  could  be  used,  suppose  we  improve  the  MTBF  of  the  main 
display  unit  from  1 ,500  hours  to  2,000  hours.  The  total  cost  will  then  be  decreased  to 
$440,319,492,  representing  approximately  $2.3  million  savings  in  maintenance  cost. 

This  is  valuable  information  for  the  program  manager  when  s/he  makes  the  component- 
reliability  improvement  decisions. 
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Figure  1.  Input  Parameter  for  Model  2 


No  of  Squadrons 

4 

No  of  UAV  systems  per  squadron 

10 

No  of  Air  Vehicles  per  system 

4 

No  of  Ground  Control  stations  per  system 

2 

Ground  Equip  Monthly  Op  Hrs  Hours 

300 

hrs 

AV  Flying  Hours/Vehicle/month 

120 

hrs 

AutoLand  &  Launch/RecMonthly  Op  Hours 

60 

hrs 

Repair  Turnaround  Time 

10 

days 

Protection  Level  for  Critical  Components 

0.95 

Protection  Level  for  non-Critical  Components 

0.85 

Hourly  charge  for  repair  including  material  cost 

$500 

Transportation  cost  per  failure 

$200 

Annual  Inventory  rate 

21% 

Capital  Discount  rate 

10% 

Lifecycle 

20 

years 

Ground  Control  Station  Components 

MTBF 

Unit  Cost 

Main  Display  Unit 

1000 

0.00100 

$  500,000 

Critical 

Power  Supply 

4000 

0.00025 

$  400,000 

Critical 

Power  Gen 

3500 

0.00029 

$  300,000 

Critical 

Air  Conditioner 

6000 

0.00017 

$  400,000 

Critical 

Guidance  &  Control 

500 

0.00200 

$  400,000 

NonCritical 

Other  Ground  Equip 

MTBF 

Unit  Cost 

Launch  &  Recovery  System 

500 

0.00200 

$  1,200,000 

Critical 

AutoLand  System 

1000 

0.00100 

$  2,000,000 

NonCritical 

Data  Terminal 

3000 

0.00033 

$  1,000,000 

NonCritical 

AV 

MTBF 

Unit  Cost 

Navigation/Avionics 

1000 

0.00100 

$  200,000 

Critical 

Engine 

500 

0.00200 

$  100,000 

Critical 

Propeller 

500 

0.00200 

$  50,000 

Critical 

Video  Scanner 

2500 

0.00040 

$  150,000 

NonCritical 

IR  Scanner 

450 

0.00222 

$  150,000 

NonCritical 

IR  Data-Link 

800 

0.00125 

$  200,000 

NonCritical 
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Figure  2.  Sample  Output  of  Model  2 


Annual  Spare  Inventory  Cost 

$  2,688,000 

per  squadron 

Annual  Repair  Cost 

$  8,800,857 

per  squadron 

Annual  Transportation  cost 

$  328,034 

per  squadron 

Total  cost  per  squadron  per  year 

$  11,816,891 

Total  Annual  cost 

$  47,267,566 

Total  Lifecycle  Cost 

$442,656,976 

Once  the  cost  analysis  is  completed  (using  Model  2),  the  same  input  parameters 
are  used  for  the  simulation  model  (Model  3)  to  estimate  the  operational  availability  and 
other  performance  measures  of  the  system  (e.g.,  probability  that  the  operational 
availability  falls  below  some  critical  level).  Model  2  and  Model  3  (simulation  model) 
complement  each  other. 

3.3.  Simulation  Scenarios 

In  this  simulation  model  (Model  3),  we  only  consider  the  critical  components 
(engine,  propeller,  avionics  computer)  for  a  squadron  of  10  UAV  systems  with  40  air 
vehicles  (see  Appendix  A).  When  one  of  these  critical  components  fails,  the  faulty 
component  is  removed  from  the  air  vehicle,  and  an  RFI  (ready-for-issue)  spare  is 
installed.  The  faulty  component  is  sent  to  the  repair  shop  to  be  fixed.  After  repair,  it 
becomes  an  RFI  spare.  When  a  critical  component  fails,  and  an  RFI  spare  is  not 
available,  the  air  vehicle  will  be  grounded  (and  will  become  not-mission  capable,  or 
NMC)  until  an  RFI  component  is  available.  A  failure  of  non-critical  components  may 
degrade  readiness,  but  the  system  is  assumed  to  be  operable  (that  is,  mission  capable 
or  MC). 

The  input  parameters — such  as  MTBF  and  number  of  spares  for  each 
component,  repair  times  (in  hours),  transportation  delay  (one  way,  in  days) — are  read 
from  the  spreadsheet  (see  Figure  3).  When  a  component  fails  in  Scenario  1 ,  it  requires 
9  days  (4.5  days  one  way)  of  transportation  delay  with  10  hours  of  repair  work;  this  work 
follows  a  triangular  distribution  with  a  mode  of  10  hours,  an  upper  limit  of  50%  above 
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the  mode  (i.e.,  15  hours)  and  a  lower  limit  of  50%  below  the  mode  (i.e.,  5  hours).  The 
waiting  time  at  the  repair  shop,  if  any,  is  estimated  inside  the  simulation.  The  repair  time 
of  10  hours  and  the  transportation  delays  of  9  days  (4.5  day  each  way)  in  Figure  1  f 
approximate  the  total  repair  TAT  of  ten  days  in  Scenario  1 . 


Figure  3.  Simulation  Input  Spreadsheet 


Scenario 

MTBF_ 

Eng 

MTBF_ 

Prop 

MTBF_ 

AvComp 

Spare 

Engines 

Spare 

Props 

Spare 

AvComps 

Eng 

Repair 

hrs 

Prop 

Repair 

hrs 

AvComp 

Repair 

hrs 

Trans 

Delays 

(Days) 

1 

1000 

500 

500 

4 

6 

6 

10 

10 

10 

4.5 

2 

1250 

500 

500 

4 

6 

6 

10 

10 

10 

4.5 

3 

1500 

500 

500 

4 

6 

6 

10 

10 

10 

4.5 

4 

1000 

750 

500 

4 

6 

6 

10 

10 

10 

4.5 

5 

1000 

1000 

500 

4 

6 

6 

10 

10 

10 

4.5 

6 

1500 

1000 

500 

4 

6 

6 

10 

10 

10 

4.5 

7 

1000 

500 

500 

10 

10 

10 

10 

10 

10 

4.5 

8 

1000 

500 

500 

4 

6 

6 

10 

10 

10 

2.25 

9 

1000 

500 

500 

4 

6 

6 

10 

10 

10 

1 

10 

1500 

1000 

500 

4 

6 

6 

10 

10 

10 

1 

Given  the  input  parameters  in  Figure  3,  Model  3  simulates  each  scenario  over 
1,000,000  hours.  Multiple  scenarios  can  be  executed  in  one  simulation  run  (e.g.,  10  in 
this  case).  The  results  captured  for  each  scenario  are  the  average  operational 
availability  (Ao)  for  the  air  vehicles  in  the  squadron,  along  with  the  cumulative 
distribution  of  operational  availability.  These  results  are  tabulated  in  Figure  4L  The 
cumulative  distribution  of  operational  availability  is  also  depicted  graphically  in  Figure  5. 
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Figure  4.  Simulation  Output:  Cumulative  Operational  Availability  and  the  Average 


Operational  Availability  for  Each  Scenario 


Cumulative  Operational 

Availability 

Avg 

Op  Av 

10% 

20% 

30% 

40% 

50% 

60% 

70% 

80% 

90% 

100% 

1 

0.00 

0.00 

0.00 

0.00 

0.00 

0.02 

2.27 

32.25 

90.91 

100.00 

0.837 

2 

0.00 

0.00 

0.00 

0.00 

0.00 

0.01 

1.68 

28.76 

88.99 

100.00 

0.843 

3 

0.00 

0.00 

0.00 

0.00 

0.00 

0.01 

1.50 

27.53 

88.47 

100.00 

0.845 

4 

0.00 

0.00 

0.00 

0.00 

0.00 

0.01 

0.95 

20.50 

84.35 

100.00 

0.857 

5 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.55 

16.50 

80.87 

100.00 

0.865 

6 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.22 

11.70 

75.25 

100.00 

0.876 

7 

0.00 

0.00 

0.00 

0.00 

0.00 

0.04 

2.27 

32.43 

91.39 

100.00 

0.837 

8 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.03 

3.14 

52.74 

100.00 

0.906 

9 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.08 

15.37 

100.00 

0.948 

10 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.02 

6.63 

100.00 

0.962 

Figure  5.  Cumulative  Operational  Availability 


Cumulative  Operational  Availability 


- Scenario  1  - Scenario  2  - Scenario  3  - Scenario  4  - Scenario  5 

- Scenario  6  - Scenario  7  - Scenario  8  - Scenario  9  - Scenario  10 
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Let’s  assume  that  the  commander’s  goal  is  to  maintain  an  average  Ao  of  85%. 

He  also  knows  his  mission  capability  will  be  critically  jeopardized  if  Ao  falls  below  80%. 
Therefore,  he  wants  to  estimate  the  probability  that  this  event  (maintaining  the  average 
Ao  to  be  above  85%)  might  happen.  The  results  in  Figure  4  show  that  the  average  Ao 
of  Scenario  1  is  83.7%  (the  last  column  of  Scenario  1 )  and  the  probability  of  Ao  falling  to 
80%  or  below  is  32.25%  (the  9th  column  with  a  heading  of  80%  for  Scenario  1 ). 

Scenario  1  is  not  acceptable  to  the  commander  since  the  average  Ao  is  below  his  goal, 
and  the  probability  of  Ao  falling  below  80%  seems  to  be  too  high.  He  can  generate 
more  scenarios  (e.g.,  Scenarios  2  through  10)  to  assess  the  impact  of  changes  in 
component  reliability  or  logistics  elements  (spare  parts,  repair  and  transportation  times) 
on  the  entire  system-level  Ao. 

In  Scenarios  2  and  3,  the  MTBF  of  an  engine  is  increased  from  1,000  hours  to 
1 ,250  and  1 ,500,  respectively.  In  Scenarios  4  and  5,  the  MTBF  of  a  propeller  is 
improved  from  500  hours  to  750  and  1 ,000,  respectively.  Improvement  in  Ao  can  be 
observed  from  the  far  right-hand  side  column  of  the  Figure  4.  Changes  in  Scenarios  4 
and  5  are  preferred  to  those  of  Scenarios  2  and  3.  In  Scenario  6,  the  MTBFs  of  both  the 
engine  and  propeller  are  increased  respectively  to  1 ,500  and  1 ,000.  The  overall  Ao  is 
increased  to  87.6%  (from  83.7%  of  Scenario  1),  and  the  probability  of  Ao  falling  below 
80%  has  substantially  reduced  to  1 1 .7%  (from  32.25%).  Increase  in  spare  parts 
(Scenario  7)  does  not  improve  the  performance  at  all.  However,  significant  reduction  in 
transportation  time  (Scenario  8)  improves  the  system  performance.  In  Scenarios  8  and 
9,  when  the  transportation  delays  are  reduced  from  4.5  days  to  2.25  and  1  respectively, 
Ao  jumps  to  90.6%  and  94.8%,  respectively;  likewise,  the  probabilities  of  Ao  falling 
below  80%  drop  to  3.14%  and  0.08%,  respectively.  The  Scenario  10  is  the  same  as 
Scenario  9  except  that  the  MTBFs  of  an  engine  and  a  propeller  are  increased  to  1 ,500 
and  1,000,  respectively.  Ao  hits  96.2%  with  the  probability  of  Ao  falling  below  80%  now 
negligible  (0.02%). 

The  parameters  in  Scenarios  2  through  10  can  be  input  to  Model  2  to  compute 
the  total  maintenance  cost  for  each  scenario.  For  example,  by  entering  the  parameters 
from  Scenario  10  into  Model  2  in  Figure  2,  a  PM  will  note  results  in  a  total  lifecycle 
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maintenance  cost  of  $375,712,781  (i.e.,  savings  of  approximately  $120  million  over  the 
base  case  of  Scenario  1).  Scenario  10  provides  an  Ao  12.5%  higher  than  Scenario  1 
(from  83.7%  to  96.25%)  with  the  risk  of  Ao  falling  below  80%  becoming  a  non-issue. 

Models  2  and  3  can  potentially  serve  as  a  communication  tool  between  the 
budget  community  and  warfighters.  When  reliability  improvements  are  made  on  several 
components  in  a  complex  system,  the  warfighter’s  primary  concern  is  readiness,  or  Ao, 
while  the  budget  analysts’  focus  is  on  financial  implications.  These  two  models  provide 
valuable  solutions  to  both  communities. 
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IV.  SUMMARY 


Providing  reduced  lifecycle  cost  and,  at  the  same  time,  improving  operational 
availability  are  fundamental  goals  of  the  Performance-based  Logistics  (PBL)  and  other 
logistics  initiatives  of  the  US  Department  of  Defense.  In  many  PBL  contracts,  the 
contractual  arrangements  are  typically  stipulated  at  the  level  of  individual  components 
(such  as  a  fuel  cell)  or  a  logistic  element  (such  as  inventory  of  certain  spare  parts). 
While  achieving  component-level  performance  goals  is  certainly  important,  what  really 
matters  to  a  warfighter  is  the  operational  availability  of  the  weapon  system.  Hence, 
there  is  a  need  to  develop  a  methodology  and  an  apparatus  for  estimating  the 
operational  availability  (Ao)  of  a  weapon  system  based  on  the  component-level 
reliability  and  maintainability  data.  This  current  research  is  aimed  at  this  need. 

Specifically,  we  present  two  spreadsheet  models  and  one  discrete-event 
simulation  model  using  Arena  simulation  language.  The  first  model  primarily  supports 
lifecycle  cost  calculations,  but  ignores  the  interactions  among  reliability,  time  to  repair, 
and  operational  availability.  The  second  model,  while  it  does  address  these  basic 
interactions,  does  not  consider  the  full  range  of  lifecycle  costs.  However,  both  the  first 
and  the  second  models  are  static — they  can  only  support  average  case  analyses  and 
sensitivity  analyses.  The  third  model  incorporates  the  interactions  among  reliability, 
time  to  repair  and  operational  availability  into  a  simulation  model  that  can  support  a 
weapon-system-level  risk  analysis.  In  their  current  form,  these  models  are  developed 
as  a  proof-of-concept.  That  is,  we  are  not  presenting  a  research  case  involving  field 
data,  but  rather  are  demonstrating  the  potential  methodology  and  a  tool  using 
hypothetical,  yet  realistic,  data. 
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VI.  APPENDIX:  Unmanned  Aerial  Vehicle  (UAV)  Case 
Study 


A  UAV  system  consists  of  four  air  vehicles  (AV's),  two  ground-control  stations 
(GCSs),  modular  mission  payloads  (MMPs),  data  links,  remote  data  terminals  (RDTs) 
and  an  automatic  landing  system.  A  total  of  8  squadrons  (two  squadrons  in  each  coast 
of  CONUS,  and  one  each  for  Pacific,  Indian,  Mediterranean,  and  Atlantic  Oceans)  will 
be  established  to  accommodate  the  new  system.  Each  squadron  will  have  its  own 
intermediate-level  maintenance  capabilities.  Each  squadron  will  have  10  VTUAV 
systems.  Detachment  personnel  (for  each  UAV  system)  will  consist  of  three  officers 
(one  OIC  and  two  mission  officers),  three  Chief  Petty  Officers  (CPOs)  and  12  enlisted. 
1-Level  Maintenance  personnel  will  consist  of  one  officer,  one  Chief  petty  officer  and  ten 
enlisted.  Squadron  headquarters  personnel  will  be  made  up  of  seven  officers,  ten  CPOs 
and  twenty  enlisted.  Composite  costs  for  personnel  are  estimated  as  follows:  Officer — 
$140,000  per  year,  CPO — $115,000  per  year,  Enlisted — $70,000  per  year. 

Production  begins  in  Fiscal  Year  2004,  with  all  VTUAV's  scheduled  for  field 
tesing  in  the  year  following  their  production.  A  total  of  80  VTUAV  systems  will  be 
produced;  the  lifecycle  of  the  program  is  estimated  to  be  30  years  (2005-2034).  The 
risk  of  loss  of  an  AV  in  peace  time  is  2-7%  per  year,  while  the  risk  of  loss  of  an  AV  in 
operation  during  a  contingency  is  15-30%  per  year.  A  chance  of  a  contingency  during 
the  lifecycle  of  the  program  is  15%  per  year.  Lost  AVs  will  be  replaced  the  next  year. 
However,  no  orders  for  replacement  AVs  will  be  placed  in  the  last  5  years  of  the 
lifecycle  (i.e.,  YR  2029  -  2034).  We  are  assuming  by  then  new  UAV  systems  will 
gradually  replace  the  current  ones. 

Research  and  development  costs  are  $15  million  for  FY  01,  $20  million  in  FY  02 
and  $50  million  in  FYs  03  and  04.  The  marginal  production  cost  of  AV  (with  payload)  is 
$1  million.  The  cost  of  maintaining  a  production  capability  throughout  the  life  of  the 
system  is  $12  M  per  year  for  every  year  any  aerial  vehicles  are  produced.  Thus,  the 
annual  production  cost  of  AV  is  $12M  +  $1M  *  (#  of  AV  produced).  Ground-Control 
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Equipment,  which  consists  of  two  GCSs,  RDTs,  test  equipment  and  an  automatic 
landing  system,  will  cost  $20  million  per  system.  The  l-level  operating  cost  is  $6 
million/yr  per  l-level  plus  an  additional  one-time  capital  investment  of  $25  million 
(including  installation  of  test  equipment)  prior  to  the  year  of  operation.  A  capital 
discount  rate  of  10%/yr  and  the  inflation  rate  of  4%/yr  will  be  used. 

Billet  requirements  are  based  on  all  personnel  fully  qualified/current/certified  to 
perform  all  missions/Navy  Enlisted  Classification  Code  (NEC)/Military  Occupational 
Specialty  (MOS).  Operators  are  required  to  have  functional  applications  of  the  use  and 
control  of  the  UAV,  and  will  be  trained  in  operation  of  all  aspects  of  the  UAV  navigation, 
launch  flight  control  and  recovery.  Officers  and  CPOs  will  attend  additional  training  on 
preflight  planning,  mission  profile  construction  and  UAV  tactical-intelligence  integration. 
Costs  for  the  training  will  be  $1 ,600/person/week  for  the  basic  training  and  $3,000/ 
person/week  for  the  advanced  training.  An  attrition  rate  of  25%  per  year  is  used  after 
the  first  year,  including  personnel  rotation.  Required  training  is  as  follows: 

Detachment  personnel 

•  Basic  UAV  Training  (Officers,  CPOs,  junior  enlisted):  10  weeks 

•  Advance  Training  (Officers  and  CPOs  only):  5  weeks 

1-Level  Maintenance  personnel 

•  Basic  Maintenance  Training  (Officers,  CPOs,  junior  enlisted):  20  weeks 

•  Advance  Maintenance  Training  (Officers  and  CPOs  only):  5  weeks 

Squadron  Headquarters  personnel 

•  Basic  UAV  intelligence  course  (Officers,  CPOs,  junior  enlisted):  10  weeks 

•  Advance  Training  (Officers  and  CPOs  only):  5  weeks 

Spare  parts  management  will  be  consolidated  at  the  1-Level  on  a  one-for-one 
exchange.  We  will  assume  that  the  transportation  cost  is  $100  per  shipment  (i.e. ,  $200 
per  failure).  Spares  replacement  and  repair  materials  cost  will  be  equal  to  50%  of  the 
value  of  spares  per  year.  Sparing  levels  will  be  as  follows:  critical  units — 95%  and  non- 
critical  units — 85%.  Maintenance  turnaround  time  (TAT),  including  transportation 
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delays,  for  1-Level  is  10  days  and  D-Level  is  40  days.  It  is  assumed  that  80%  of  failures 
can  be  repaired  at  the  1-Level  (thus  20%  at  the  D-level).  Spare-level  calculations  are 
based  on  "t  =10  (0.8)  +  40  (0.2)  =  16  days."  D-Level  cost  is  estimated  to  be  $5,000  per 
repair  including  the  transportation  costs.  Ground  equipment  is  expected  to  operate  300 
hours  per  month;  the  AV  flying  hour  is  estimated  at  120  hours  per  month  per  vehicle. 
The  launch/recovery  and  the  auto-landing  systems  are  used  20%  of  the  time  the 
ground-control  station  is  in  operation  (i.e.,  60  hours  per  month).  POL  (petroleum,  oil  and 
lubricant)  costs  are  estimated  at  $60  per  flight  hour.  The  MTBF  of  each  component,  its 
cost,  and  the  required  protection  level  (customer  service  level)  are  included  as  follows: 


MTBF 

Cost 

Criticality 

1.  Ground  Station  (2  perVTUAV  system) 

Main  Display  Unit 

1,500  hrs 

$ 

500,000 

critical 

Power  Supply 

4,000  hrs 

$ 

400,000 

critical 

Power  Generator 

3,500  hrs 

$ 

300,000 

critical 

Air  Conditioner 

6,000  hrs 

$ 

400,000 

critical 

Guidance  &  Control 

500  hrs 

$ 

400,000 

non-critical 

II.  Other  Ground  Equipment  (1  perVTUAV  system) 

Launch/Recovery  System 

500  hrs 

$1,200,000 

critical 

Auto-landing  System 

1 ,000  hrs 

$2,000,000 

non-critical 

Data  Terminal 

3,000  hrs 

$1,000,000 

non-critical 

III.  AV  and  Pavload 

Engine 

500  hrs 

$ 

100,000 

critical 

Propeller 

500  hrs 

$ 

50,000 

critical 

Navigation/avionics 

1,000  hrs 

$ 

200,000 

critical 

Video  Scanner 

2,500  hrs 

$ 

150,000 

non-critical 
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IR  Scanner 


450  hrs  $  150,000  non-critical 
800  hrs  $  200,000  non-critical 


IR  Data-Link 

The  System  activation/deactivation  plan  is  as  follows: 

System  Activation  plan:  FY  2005  -  20  systems 
(2  squadrons  at  a  time)  FY  2006  -  20  systems 

FY  2007  -  20  systems 
FY  2008  -  20  systems 

System  Deactivation:  FY  2031  -  20  systems 

(phase-out)  plan  FY  2032  -  20  systems 

(2  squadrons  at  a  time)  FY  2033  -  20  systems 

FY  2034  -  20  systems 
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